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Abstract. This note is an attempt to give an answer for the following 
old I.M. Gelfand's question: why some important problems of integral 
geometry (e.g., the Radon transform and others) are related to harmonic 
analysis on groups but for other quite similar problems such relations are 
not clear? In the note we examine standard problems of integral geome- 
try generating harmonic analysis (the Plancherel theorem etc.) on pairs 
of commutative hypergroups that are in a duality of Pontryagin's type. 
As a result new meaningful examples of hypergroups are constructed. 



The subject of integral geometry (in the sense of [H[2j[Tl]) is formed 
by integral transforms mapping functions on a manifold (or space) X 
to their integrals over submanifolds in X forming a family X of sub- 
manifolds in X, so new functions defined on X appear. One of basic 
problems in integral geometry is a reconstruction of the initial function 
on X starting from its image on X. 

This note is an attempt to give an answer for the following old 
I. M. Gelfand's question: why some important problems of integral ge- 
ometry (e.g., the Radon transform and others) are related to harmonic 
analysis on groups but for other quite similar problems such relations 
are not clear? In the note we examine standard problems of integral ge- 
ometry generating harmonic analysis (the Plancherel theorem etc.) on 
pairs of commutative hypergroups that are in a duality of Pontryagin's. 
As a result new meaningful examples of hypergroups are constructed. 

1. Hypergroups and generalized Fourier transforms. Suppose 
that X and X have structures of smooth manifolds or diffeological 
spaces with fixed measures dx and dy respectively. 

Let us examine transforms of the form 

F : f(x) H> f{y) = J f(x)e(x,y) dx, (1) 

where e(x, y) is a generalized function on X x X. Suppose that this 
transform generates an isomorphism L 2 (X, dx) — > L 2 (X, dy), such that 
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the following generalized Plancherel formula is valid: 

J f{x)g{x)dx = J f{y)g{y)dy. (2) 

In addition suppose that this isomorphism can be extended to 5- 
functions and set g(x) = S x (-); then from (j2J) it follows that 5 X = e(x, y) 
and the following inversion formula is valid: 

f( x ) = J f(y)e(x,y)dy. (3) 

On the other hand the Plancherel formula ([2]) follows from (pQ) and 
03]). We suppose that in the space L 2 (X,dx) there is a dense locally 
convex linear subspace S (or lineal for the sake of brevity) , such that 
S consists of continuous functions (but not all of them) and the lineal 
S = F(S) possesses the same properties with respect to the space 
L 2 (X,dy). It is assumed that S and S are algebras with respect to 
the usual multiplication of functions and the isomorphism ([T]) can be 

extended to (5-functions belonging to the spaces M and M, dual to S 
and S respectively. 

The described construction is a formalization and generalization of 
heuristic ideas of B. M. Levitan [5j [6]. 

In the case at hand, when formulas ([I]) and ([3]) and hence ([2]) are 
valid, we shall say that the transform F is a generalized Fourier trans- 
form (or GFT for the sake of brevity); functions e(x, y) and e(x, y) will 
be called generalized exponential functions. 

Proposition 1. Under the specified assumptions, X has a structure of 
commutative hypergroup in the following sense: generalized translation 
operators act in S and for these operators the associativity axiom of J. 
Delsarte [1] is valid; generally speaking, it is not supposed that X has a 
neutral element. These generalized translation operators are defined by 
the formula 

BP f{x) = J f{x) ■ e(y, X ) • eix, X )d X , (4) 

where x £ X defines the character x( x ) = e ( x > X) on X. Similarly 
x (x) — e ( x i x) is a character on X. 

The hypergroup commutativity means that every two generalized 
translation operators commute. Then X is a hypergroup dual to X; of 
course, the hypergroup X is dual to X and F _1 (5j / ) = e(x, y). 

For example, if X = X = M n , dx and dy are normalized invariant 
measures, then F is the usual Fourier transform, e(x,y) is the expo- 
nential function e l( - x ' y \ and for S it is possible to use the L. Schwartz 
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space of functions which are rapidly decreasing with all the deriva- 
tives. In this case the space M consists of all tempered generalized 
functions (distributions). Below in Section 5 the space S is the space 
of all smooth functions on a compact hypergroup X; in other sections 
S is the L. Schwartz space on a hypergroup X. 

Usually generalized translation operators act also in the spaces of 
smooth functions, summable functions, measures and generalized func- 
tions with compact supports etc. Different versions of the concept of 
hypergroup are discussed, e.g., in [I] -[12]. Of course, every group (or 
semigroup) G is an example of a hypergroup; in this case translations 
R x are of the form: 

R* : f(t) H- f(tx), 

where tx is the product of elements of the group. 

There are quite few examples of nontrivial meaningful hypergroups 
related to harmonic analysis. The most well known one is the Gelfand 
pair related to harmonic analysis of spherical functions, see, e.g., [12116]. 
One of our aims in this note is to extend the collection of examples of 
this type. 

2. GFT associated with the generalized Radon transform 

on C n . The Radon transform on C" maps functions / on C n to their 
integrals over hyperplanes in C n , i.e. functions Oif on the manifold of 
hyperplanes in C n . For an explicit description of this transform we shall 
define hyperplanes by the equations x n = Y^h=i a i x i + a n an d we shall 
use a = (ai, . . . , a n ) as local coordinates for the manifold of hyperplanes 
in C n . Using these notations and the delta-functions notation we can 
render the Radon transform in the following form: 

(ft/)(a) = / f(x)6(x n -aiXi-...-a n -ix n -i-a n )dn(x), 

where 5(-) is the delta-function on C , and dp,(x) is the Lebesgue mea- 
sure on C n . It is known that the following inversion formula is valid: if 
tp = "Jlf, then 

f(x) = / v?(a)5 (n_1 ' n_1) (x„ - a^i - . . . a n _ix„_i - a n )dfi(a), 

where S^^\t) = ^g^r^i), 

We shall say that a generalized Radon transform on C n associated 
with an arbitrary generalized function u(t) on C is the integral trans- 
form 

(R u f)(a) = / f(x)u(x n - aixi - . . . - a n -xx n -\ - a n )dn(x). 
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Theorem 1. If the Fourier transform u(c) of u(t) is an usual function 
and this function is nonzero almost everywhere, then the generalized 
Radon transform Jl u is invertible and the inversion formula is of the 
form^: if Lp = J u f, then 

f(x)= / ip(a)U(x n - atxi - . . .a„_ix n _i - a n )d/J,(a), (5) 
where U(t) = f c [^(c)]- 1 ^ 2 ^- V Re ( c ')d/i(c). 

Corollary. If \u(c)\ = |c| n_1 , then the generalized Radon transform J u 
is a GFT, %.. U(t) =u(tj. 

Examples of GFT. Functions u(c) = c fc c n_fc_1 , k = 0, 1, . . . , n — 1 
correspond to local GFT with kernels u(t) = 5( k ' n ~ h ~ 1 \t). Functions 
u(c) = c A c M , which differ from functions indicated above, where Re(A + 
/i) = n — 1 and A — /i is an integer number, correspond to GFT with 
nonlocal kernels u{t) = t~ x ~H 11 1 . 

3. GFT associated with generalized Radon transform on M n . 
Definitions of the usual (see [3]) and generalized Radon transform on 
M n are similar to the complex case. In the real case the generalized 
Radon transform "Jl is a GFT if the Fourier transform u(c) of the ker- 

n — 1 

nel u(t) satisfies the equation \u(c)\ = \c\~z~ . Hence (see [13]) we have 
one and only one local GFT for odd n, n = 2k + 1 only. This ker- 
nel corresponds to the function u(c) = c k , and its kernel has the form 
S k (t). Examples of GFT with nonlocal kernels are GFT with kernels 

77-1-1 

u (t) = \t\-2'p, P e R. 

Remark. Similarly it is possible to construct GFT for generalized 
Radon transforms on L n , where L is a continuos non- Archimedean 
locally compact field. 

4. GFT associated with a transform of functions on the 
sphere S n C R n+1 . Let us examine an integral transform mapping 
even functions on the sphere S n to their integrals on geodesic hyper- 
surfaces (analogs of big circles on S 2 ). In the spherical coordinates 
uj these hypersurfaces are defined by the equations (£,w)=0, and the 
integral transform can be presented in the form 

/M ->• (Jf)(0 = ! /(w)tf((e,w»dw, (6) 



^Here and in the next integral formulas we suggest that coefficients which usually are 
placed before the integral symbol are included in the corresponding measure normalization. 
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where du is an invariant measure on the sphere. There exists an inver- 
sion formula representing the initial function / via its image under the 
transform Jf. 

For every A G C let us define the following generalized transform for 
even functions on S n by the equation 

Wm = [ /H^^dw. (7) 

The integral converges for Re A > — 1 and it is defined for every A G C 
as an analytic continuation in A. In particular, J_i = J. 

Theorem 2. The following inversion formula holds: 

V = J\f, f = J-\-n-i<p. (8) 

Corollary. The transform J\ is a GFT for A = — + ip, p G M . 

In particular, for n = 4k + 1 and A = —2k — 1 we have the GFT with 
the local kernel 5 2k (-). 

Similarly define the transform J u for odd functions on the sphere re- 
placing |(£,u;)| A with |(£,u;)| A sgn((£, cj)) in ([7]). For this definition, the 
same inversion formula (jSj) holds and the transform J u is a GFT under 
the same conditions as in the case of even functions. The difference is 
that local GFT exist in the case of n = 4k — 1, and in this case the ker- 
nel is the odd generalized function 5 2fc_1 (-). Note that our hypergroup 
is compact and the dual hypergroupm is compact too. 

5. GFT related to the complex of A>dimensional planes in 
C n . Define a family of n- dimensional sub manifolds (complexes) in the 
manifold of all fc-dimensional planes in the space C n ,0<A;<n— 1. 

Let the space C n be represented in the form of direct sum C n = 
C k ®C l , k + I = n of the spaces with coordinates x = {x l: . . . , x k ) and 
y = (yi, . . . , ui). Define k- dimensional planes in C n by the following 
equations solved with respect to the coordinates yi '■ yi — UnXi + . . . + 
u ik x k + o/i, i — 1, . . . , I. 

Let us fix an arbitrary (/, fc)-matrix u(t) = \\uij(t)\\ whose elements 
are polynomials of t = (£i, . . . , t&) G C k . For this matrix let us con- 
struct the submanifold of fc-dimensional planes defined by the equations 

yi = u il (t)x 1 + . . . + u ik (t)x k + a>i, i = l,...,l, (9) 

or y = u(t)x + a for short. 

The condition that the submanifold K just defined is a complex, i.e. 
dim K = n, is equivalent to the condition of nonsingularity for the map 
t i — y u(t) of the space C k to the space of (I, /c)-matrices. We shall use 
vectors a = (ax, ...,«;) and t — (tx, ■ ■ ■ , t k ) as coordinates for K. 
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Complexes K have a simple geometric structure. Specifically, equa- 
tions for a = define a /c-dimensional family of (k — l)-dimensional 
planes on a hyperpane in C n at infinity. The complex K consists 
of all A;- dimensional planes containing at least one of these (k — 1)- 
dimensional planes. In particular, for k = 1 the complex K consists 
of straight lines in C n that intersec a fixed curve in a hyperplane at 
infinity. 

The complex K generates an integral transform J mapping functions 
f(x,y) on C n to their integrals over planes of the complex. Using the 
delta-functions notation, it is convenient to present this transform in 
the form 

(Jf)(a,t)= f(x,y)S{y-u(t)x-a)dn(x)dfi(y), (10) 
Jc n 

where 5(-) is the delta-function on C fc , and dfi(x), dfi(y) are Lebesgue 
measures on C k and C 1 . 

We now examine general transforms J a which are defined by replac- 
ing in definition (Tl0|) the delta-functions 5(s) with arbitrary generalized 
functions a(s,t) and find inversion formulas for these transforms. By 
definition we have 

(J a f)(a,t) = J f(x,y)a(y-u(t)x- a,t)d/j,(x)d(j,{y). (11) 

Note that J a f is a result of convolution with respect to a of the function 
(p — Jf with the function a(s,t) : 

{JJ)(a,t)= [ ip(a + 8,t)a(s,t)dfjL(s). (12) 

There is a simple relation between the Fourier transform f (?),$,) of 
the function f(x,y) and the Fourier transform tp(£,t) with respect to 
a of the function tp = J a f : 

${Z,t) = f(-£<t),t)a(-Z,t), (13) 
where a(£, t) is the Fourier transform with respect to s of the function 
a(s, t). 

Let us check that ifa(£,i) is an usual function which is nonzero 
almost everywhere, then the function f can be reconstructed from the 
function ip = J a f in a unique way. 

Note that by virtue of (TT5|) this condition implies that the func- 
tion F(rj,£) = f(—£u(t),£) can be reconstructed from the function tp 
uniquely. On the other hand from the nonsingularity of K and the ana- 
lyticity condition it follows that for almost every pair (77, £) there exists 
t G C k such that —£u(t) = n. It then follows that the function /, and 
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hence the function / can be reconstructed from the function F. Let us 
describe an inversion formula explicitly. 

Definition 1. We shall say that the Crofton function related to the 
complex K, is the function Cvk(Vi 011 C k © C , equal to the number 
of solutions t of the equation r] = —£u(t). If the Crofton function is 
constant almost everywhere, then this constant is called the Crofton 
number. Denote this number by Cr^ . 

From the condition on the complex K it follows that its Crofton 
function is nonzero and constant almost everywhere. 

Theorem 3. // a(£, t) is a usual function which is nonzero almost 
everywhere, then there is the following inversion formula for the integral 
transform f 1— )■ (p = J a f : 



where 



f(x,y) 



A(s,t) 



f(a, t) A(y — u(t)x — a, t) dfi(a) d/J,(t) 



(14) 



Cr* 



dt 



Corollary. Under the conditions of the theorem [3] the integral trans- 
form J a is an GFT iff the Fourier transform a(£, t) with respect to s of 
the kernel a(s, t) satisfies the equation 



a(£,t)\ = Cr K 1/2 \u(Z,t)\, 



dt 



(16) 



Examples of GFT are the integral transforms J a with the kernels 
a(s, t), for which the Fourier transforms with respect to s are functions 
a(£, t) of the form 



% t) = Cr 



-1/2 



A' 



X p eC 



(17) 



P =i 

Let us describe the kernels a(s, t) explicitly. Since t) is a homo- 
geneous polynom of £1, . . . , of the degree k, then the function a(£, t) 
can be presented in the form 

1 



E 

mi + ...+m;=fc 



U 



mi,..., mi 



(*) II(e 



m„+A, 



p=i 



Therefore 



a(s, £) 



mi+...+m;=fc 



mi,...,m; 



p=i 



8 



MX GRAEV, GX. LITVINOV 



where J is the inverse Fourier transform. In particular, (see [2]), if 
\p 7^ 0, p = 1, . . . , I, then 3 r (£™ p+Ap £ p P ) is, up to a factor, the fuction 

— nip— \p — 1 A p — 1 

Sp Sp 

In the special case Ai = . . . = A; = the GFT is local and its kernel 
a(s, t) has the form: 

a(s,t) = Cr- 1/2 u(-^-;t)5(s). 

6. GFT related to the complex of /c-dimensional planes in 

IR n . Consider a real analog of the complex of fc-dimensional planes. 
Planes of the corresponding complex K are defined by the same equa- 
tions Q9J), where a = (ai,...,aj) G R l and t = (ti,...,t k ) G R k , 
and generalized integral transforms / i— )■ <p = J a f are defined by the 
formula fTTTT) . where a(s, £) is a generalized function onR™ and integra- 
tion is performed respect to the Lebesgue measure on M n . The same 
relation (I13p holds between the Fourier transform of / and the Fourier 
transform of ip = J a f. 

Unlike the complex case, the Crofton function for the real complex 
K is not constant in general and it can be zero on an open set. The 
transform J a can be invertible iff this function is nonzero almost every- 
where. 

Theorem 4. If the Crofton function Cr k{t), £) for the real K is finite 
and nonzero almost everywhere and the kernel a(£, t) is a usual func- 
tion which is nonzero almost everywhere, then the following inversion 
formula holds for the integral transform f y tp = J a f ; 



f(x,y) = j ip{a,t)A{y — u{t)x — a,t)dfx(a)dfi(t), (18) 

where A(s,t)= [ ^f^L,* * ^ W 

J Cr K (-£u(t),Qa(£,t) 

Corollary. The integral transform J a is an GFT if the Fourier trans- 
form a(£, t) with respect to s of the kernel a(s, t) satisfies the equation 

\a^t)\=CT K 1/2 (Cu(t),OHC,t)\ 1/2 . (19) 

In particular, if all the elements of the matrix u(t) are linear func- 
tions, then the Crofton function is identically equal to unity, hence 

\a(Z,t)\ = \u,(£,t)\ 1/2 - 
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Example. 



In the case k — 1, Crx(^,0 = 1 an d 
|V2+ip > p e r the GFT kernel has the form: 



a(s; i) = 



-3/2-ip 



n-1 
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